Rapid identification of inflammatory arthritis and associated adverse events following immune checkpoint therapy: a machine learning approach.

Introduction: Immune checkpoint inhibitor-induced inflammatory arthritis (ICI-IA) poses a major clinical challenge to ICI therapy for cancer, with 13% of cases halting ICI therapy and ICI-IA being difficult to identify for timely referral to a rheumatologist. The objective of this study was to rapidly identify ICI-IA patients in clinical data and assess associated immune-related adverse events (irAEs) and risk factors. Methods: We conducted a retrospective study of the electronic health records (EHRs) of 89 patients who developed ICI-IA out of 2451 cancer patients who received ICI therapy at Northwestern University between March 2011 to January 2021. Logistic regression and random forest machine learning models were trained on all EHR diagnoses, labs, medications, and procedures to identify ICI-IA patients and EHR codes indicating ICI-IA. Multivariate logistic regression was then used to test associations between ICI-IA and cancer type, ICI regimen, and comorbid irAEs. Results: Logistic regression and random forest models identified ICI-IA patients with accuracies of 0.79 and 0.80, respectively. Key EHR features from the random forest model included ICI-IA relevant features (joint pain, steroid prescription, rheumatoid factor tests) and features suggesting comorbid irAEs (thyroid function tests, pruritus, triamcinolone prescription). Compared to 871 adjudicated ICI patients who did not develop arthritis, ICI-IA patients had higher odds of developing cutaneous (odds ratio [OR]=2.66; 95% Confidence Interval [CI] 1.63-4.35), endocrine (OR=2.09; 95% CI 1.15-3.80), or gastrointestinal (OR=2.88; 95% CI 1.76-4.72) irAEs adjusting for demographics, cancer type, and ICI regimen. Melanoma (OR=1.99; 95% CI 1.08-3.65) and renal cell carcinoma (OR=2.03; 95% CI 1.06-3.84) patients were more likely to develop ICI-IA compared to lung cancer patients. Patients on nivolumab+ipilimumab were more likely to develop ICI-IA compared to patients on pembrolizumab (OR=1.86; 95% CI 1.01-3.43). Discussion: Our machine learning models rapidly identified patients with ICI-IA in EHR data and elucidated clinical features indicative of comorbid irAEs. Patients with ICI-IA were significantly more likely to also develop cutaneous, endocrine, and gastrointestinal irAEs during their clinical course compared to ICI therapy patients without ICI-IA.

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record.

BACKGROUND: Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR. OBJECTIVES: This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic. METHODS: The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software. RESULTS: The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system. CONCLUSION: This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Using Machine Learning to Predict Antidepressant Treatment Outcome From Electronic Health Records.

Objective: To evaluate if a machine learning approach can accurately predict antidepressant treatment outcome using electronic health records (EHRs) from patients with depression. Method: This study examined 808 patients with depression at a New York City-based outpatient mental health clinic between June 13, 2016 and June 22, 2020. Antidepressant treatment outcome was defined based on trend in depression symptom severity over time and was categorized as either "Recovering" or "Worsening" (i.e., non-Recovering), measured by the slope of individual-level Patient Health Questionnaire-9 (PHQ-9) score trajectory spanning 6 months following treatment initiation. A patient was designated as "Recovering" if the slope is less than 0 and as "Worsening" if the slope was no less than 0. Multiple machine learning (ML) models including L2 norm regularized Logistic Regression, Naive Bayes, Random Forest, and Gradient Boosting Decision Tree (GBDT) were used to predict treatment outcome based on additional data from EHRs, including demographics and diagnoses. Shapley Additive Explanations were applied to identify the most important predictors. Results: The GBDT achieved the best results of predicting "Recovering" (AUC: 0.7654 ± 0.0227; precision: 0.6002 ± 0.0215; recall: 0.5131 ± 0.0336). When excluding patients with low PHQ-9 scores (<10) at baseline, the results of predicting "Recovering" (AUC: 0.7254 ± 0.0218; precision: 0.5392 ± 0.0437; recall: 0.4431 ± 0.0513) were obtained. Prior diagnosis of anxiety, psychotherapy, recurrent depression, and baseline depression symptom severity were strong predictors. Conclusions: The results demonstrate the potential utility of using ML in longitudinal EHRs to predict antidepressant treatment outcome. Our predictive tool holds the promise to accelerate personalized medical management in patients with psychiatric illnesses.

Comparing the effects of four common drug classes on the progression of mild cognitive impairment to dementia using electronic health records.

The objective of this study was to investigate the potential association between the use of four frequently prescribed drug classes, namely antihypertensive drugs, statins, selective serotonin reuptake inhibitors, and proton-pump inhibitors, and the likelihood of disease progression from mild cognitive impairment (MCI) to dementia using electronic health records (EHRs). We conducted a retrospective cohort study using observational EHRs from a cohort of approximately 2 million patients seen at a large, multi-specialty urban academic medical center in New York City, USA between 2008 and 2020 to automatically emulate the randomized controlled trials. For each drug class, two exposure groups were identified based on the prescription orders documented in the EHRs following their MCI diagnosis. During follow-up, we measured drug efficacy based on the incidence of dementia and estimated the average treatment effect (ATE) of various drugs. To ensure the robustness of our findings, we confirmed the ATE estimates via bootstrapping and presented associated 95% confidence intervals (CIs). Our analysis identified 14,269 MCI patients, among whom 2501 (17.5%) progressed to dementia. Using average treatment estimation and bootstrapping confirmation, we observed that drugs including rosuvastatin (ATE = - 0.0140 [- 0.0191, - 0.0088], p value < 0.001), citalopram (ATE = - 0.1128 [- 0.125, - 0.1005], p value < 0.001), escitalopram (ATE = - 0.0560 [- 0.0615, - 0.0506], p value < 0.001), and omeprazole (ATE = - 0.0201 [- 0.0299, - 0.0103], p value < 0.001) have a statistically significant association in slowing the progression from MCI to dementia. The findings from this study support the commonly prescribed drugs in altering the progression from MCI to dementia and warrant further investigation.

Multi-ancestry genome- and phenome-wide association studies of diverticular disease in electronic health records with natural language processing enriched phenotyping algorithm.

OBJECTIVE: Diverticular disease (DD) is one of the most prevalent conditions encountered by gastroenterologists, affecting ~50% of Americans before the age of 60. Our aim was to identify genetic risk variants and clinical phenotypes associated with DD, leveraging multiple electronic health record (EHR) data sources of 91,166 multi-ancestry participants with a Natural Language Processing (NLP) technique. MATERIALS AND METHODS: We developed a NLP-enriched phenotyping algorithm that incorporated colonoscopy or abdominal imaging reports to identify patients with diverticulosis and diverticulitis from multicenter EHRs. We performed genome-wide association studies (GWAS) of DD in European, African and multi-ancestry participants, followed by phenome-wide association studies (PheWAS) of the risk variants to identify their potential comorbid/pleiotropic effects in clinical phenotypes. RESULTS: Our developed algorithm showed a significant improvement in patient classification performance for DD analysis (algorithm PPVs ≥ 0.94), with up to a 3.5 fold increase in terms of the number of identified patients than the traditional method. Ancestry-stratified analyses of diverticulosis and diverticulitis of the identified subjects replicated the well-established associations between ARHGAP15 loci with DD, showing overall intensified GWAS signals in diverticulitis patients compared to diverticulosis patients. Our PheWAS analyses identified significant associations between the DD GWAS variants and circulatory system, genitourinary, and neoplastic EHR phenotypes. DISCUSSION: As the first multi-ancestry GWAS-PheWAS study, we showcased that heterogenous EHR data can be mapped through an integrative analytical pipeline and reveal significant genotype-phenotype associations with clinical interpretation. CONCLUSION: A systematic framework to process unstructured EHR data with NLP could advance a deep and scalable phenotyping for better patient identification and facilitate etiological investigation of a disease with multilayered data.

Implementation of a culturally competent APOL1 genetic testing programme into living donor evaluation: A two-site, non-randomised, pre-post trial design.

INTRODUCTION: While living donor (LD) kidney transplantation is the optimal treatment for patients with kidney failure, LDs assume a higher risk of future kidney failure themselves. LDs of African ancestry have an even greater risk of kidney failure post-donation than White LDs. Because evidence suggests that Apolipoprotein L1 (APOL1) risk variants contribute to this greater risk, transplant nephrologists are increasingly using APOL1 genetic testing to evaluate LD candidates of African ancestry. However, nephrologists do not consistently perform genetic counselling with LD candidates about APOL1 due to a lack of knowledge and skill in counselling. Without proper counselling, APOL1 testing will magnify LD candidates' decisional conflict about donating, jeopardising their informed consent. Given cultural concerns about genetic testing among people of African ancestry, protecting LD candidates' safety is essential to improve informed decisions about donating. Clinical 'chatbots', mobile apps that provide genetic information to patients, can improve informed treatment decisions. No chatbot on APOL1 is available and no nephrologist training programmes are available to provide culturally competent counselling to LDs about APOL1. Given the shortage of genetic counsellors, increasing nephrologists' genetic literacy is critical to integrating genetic testing into practice. METHODS AND ANALYSIS: Using a non-randomised, pre-post trial design in two transplant centres (Chicago, IL, and Washington, DC), we will evaluate the effectiveness of culturally competent APOL1 testing, chatbot and counselling on LD candidates' decisional conflict about donating, preparedness for decision-making, willingness to donate and satisfaction with informed consent and longitudinally evaluate the implementation of this intervention into clinical practice using the Reach, Effectiveness, Adoption, Implementation and Maintenance framework. ETHICS AND DISSEMINATION: This study will create a model for APOL1 testing of LDs of African ancestry, which can be implemented nationally via implementation science approaches. APOL1 will serve as a model for integrating culturally competent genetic testing into transplant and other practices to improve informed consent. This study involves human participants and was approved by Northwestern University IRB (STU00214038). Participants gave informed consent to participate in the study before taking part. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT04910867. Registered 8 May 2021, https://register. CLINICALTRIALS: gov/prs/app/action/SelectProtocol?sid=S000AWZ6&selectaction=Edit&uid=U0001PPF&ts=7&cx=-8jv7m2 ClinicalTrials.gov Identifier: NCT04999436. Registered 5 November 2021, https://register. CLINICALTRIALS: gov/prs/app/action/SelectProtocol?sid=S000AYWW&selectaction=Edit&uid=U0001PPF&ts=11&cx=9tny7v.

Challenges of Integrating APOL1 Genetic Test Results into the Electronic Health Record.

OBJECTIVES: Integrating genetic test results into the electronic health record (EHR) is essential for integrating genetic testing into clinical practice. This article describes the organizational challenges of integrating discrete apolipoprotein L1 (APOL1) genetic test results into the EHR for a research study on culturally sensitive genetic counseling for living kidney donors. METHODS: We convened a multidisciplinary team across three institutions (Northwestern University, Northwestern Memorial HealthCare [NMHC], and OHSU Knight Diagnostic Laboratories [KDL]), including researchers, physicians, clinical information technology, and project management. Through a series of meetings over a year between the team and the genetic testing laboratory, we explored and adjusted our EHR integration plan based on regulatory and budgetary constraints. RESULTS: Our original proposal was to transmit results from KDL to NMHC as structured data sent via Health Level Seven (HL7) v2 message. This was ultimately deemed infeasible given the time and resources required to establish the interface, and the low number of samples to be processed for the study (n = 316). We next explored the use of Epic's Care Everywhere interoperability platform, but learned it was not possible as a laboratory test ordered for a research study; even though our intent was to study the APOL1 genetic test result's clinical use and impact, test results were still considered "research results." Faced with two remaining options-downloading a PDF from the KDL laboratory portal or scanning a faxed result from KDL-only a PDF of the APOL1 test result could be integrated into the EHR, reinforcing the status quo. CONCLUSION: Even with early and ongoing stakeholder engagement, dedicated project management, and funding, unanticipated implementation challenges-especially for research projects-can result in drastic design tradeoffs.

Evaluation of the portability of computable phenotypes with natural language processing in the eMERGE network.

The electronic Medical Records and Genomics (eMERGE) Network assessed the feasibility of deploying portable phenotype rule-based algorithms with natural language processing (NLP) components added to improve performance of existing algorithms using electronic health records (EHRs). Based on scientific merit and predicted difficulty, eMERGE selected six existing phenotypes to enhance with NLP. We assessed performance, portability, and ease of use. We summarized lessons learned by: (1) challenges; (2) best practices to address challenges based on existing evidence and/or eMERGE experience; and (3) opportunities for future research. Adding NLP resulted in improved, or the same, precision and/or recall for all but one algorithm. Portability, phenotyping workflow/process, and technology were major themes. With NLP, development and validation took longer. Besides portability of NLP technology and algorithm replicability, factors to ensure success include privacy protection, technical infrastructure setup, intellectual property agreement, and efficient communication. Workflow improvements can improve communication and reduce implementation time. NLP performance varied mainly due to clinical document heterogeneity; therefore, we suggest using semi-structured notes, comprehensive documentation, and customization options. NLP portability is possible with improved phenotype algorithm performance, but careful planning and architecture of the algorithms is essential to support local customizations.

Developing a standardized but extendable framework to increase the findability of infectious disease datasets.

Biomedical datasets are increasing in size, stored in many repositories, and face challenges in FAIRness (findability, accessibility, interoperability, reusability). As a Consortium of infectious disease researchers from 15 Centers, we aim to adopt open science practices to promote transparency, encourage reproducibility, and accelerate research advances through data reuse. To improve FAIRness of our datasets and computational tools, we evaluated metadata standards across established biomedical data repositories. The vast majority do not adhere to a single standard, such as Schema.org, which is widely-adopted by generalist repositories. Consequently, datasets in these repositories are not findable in aggregation projects like Google Dataset Search. We alleviated this gap by creating a reusable metadata schema based on Schema.org and catalogued nearly 400 datasets and computational tools we collected. The approach is easily reusable to create schemas interoperable with community standards, but customized to a particular context. Our approach enabled data discovery, increased the reusability of datasets from a large research consortium, and accelerated research. Lastly, we discuss ongoing challenges with FAIRness beyond discoverability.

Prediction of left ventricular ejection fraction changes in heart failure patients using machine learning and electronic health records: a multi-site study.

Left ventricular ejection fraction (EF) is a key measure in the diagnosis and treatment of heart failure (HF) and many patients experience changes in EF overtime. Large-scale analysis of longitudinal changes in EF using electronic health records (EHRs) is limited. In a multi-site retrospective study using EHR data from three academic medical centers, we investigated longitudinal changes in EF measurements in patients diagnosed with HF. We observed significant variations in baseline characteristics and longitudinal EF change behavior of the HF cohorts from a previous study that is based on HF registry data. Data gathered from this longitudinal study were used to develop multiple machine learning models to predict changes in ejection fraction measurements in HF patients. Across all three sites, we observed higher performance in predicting EF increase over a 1-year duration, with similarly higher performance predicting an EF increase of 30% from baseline compared to lower percentage increases. In predicting EF decrease we found moderate to high performance with low confidence for various models. Among various machine learning models, XGBoost was the best performing model for predicting EF changes. Across the three sites, the XGBoost model had an F1-score of 87.2, 89.9, and 88.6 and AUC of 0.83, 0.87, and 0.90 in predicting a 30% increase in EF, and had an F1-score of 95.0, 90.6, 90.1 and AUC of 0.54, 0.56, 0.68 in predicting a 30% decrease in EF. Among features that contribute to predicting EF changes, baseline ejection fraction measurement, age, gender, and heart diseases were found to be statistically significant.

Returning integrated genomic risk and clinical recommendations: The eMERGE study.

PURPOSE: Assessing the risk of common, complex diseases requires consideration of clinical risk factors as well as monogenic and polygenic risks, which in turn may be reflected in family history. Returning risks to individuals and providers may influence preventive care or use of prophylactic therapies for those individuals at high genetic risk. METHODS: To enable integrated genetic risk assessment, the eMERGE (electronic MEdical Records and GEnomics) network is enrolling 25,000 diverse individuals in a prospective cohort study across 10 sites. The network developed methods to return cross-ancestry polygenic risk scores, monogenic risks, family history, and clinical risk assessments via a genome-informed risk assessment (GIRA) report and will assess uptake of care recommendations after return of results. RESULTS: GIRAs include summary care recommendations for 11 conditions, education pages, and clinical laboratory reports. The return of high-risk GIRA to individuals and providers includes guidelines for care and lifestyle recommendations. Assembling the GIRA required infrastructure and workflows for ingesting and presenting content from multiple sources. Recruitment began in February 2022. CONCLUSION: Return of a novel report for communicating monogenic, polygenic, and family history-based risk factors will inform the benefits of integrated genetic risk assessment for routine health care.

Characterizing variability of electronic health record-driven phenotype definitions.

OBJECTIVE: The aim of this study was to analyze a publicly available sample of rule-based phenotype definitions to characterize and evaluate the variability of logical constructs used. MATERIALS AND METHODS: A sample of 33 preexisting phenotype definitions used in research that are represented using Fast Healthcare Interoperability Resources and Clinical Quality Language (CQL) was analyzed using automated analysis of the computable representation of the CQL libraries. RESULTS: Most of the phenotype definitions include narrative descriptions and flowcharts, while few provide pseudocode or executable artifacts. Most use 4 or fewer medical terminologies. The number of codes used ranges from 5 to 6865, and value sets from 1 to 19. We found that the most common expressions used were literal, data, and logical expressions. Aggregate and arithmetic expressions are the least common. Expression depth ranges from 4 to 27. DISCUSSION: Despite the range of conditions, we found that all of the phenotype definitions consisted of logical criteria, representing both clinical and operational logic, and tabular data, consisting of codes from standard terminologies and keywords for natural language processing. The total number and variety of expressions are low, which may be to simplify implementation, or authors may limit complexity due to data availability constraints. CONCLUSIONS: The phenotype definitions analyzed show significant variation in specific logical, arithmetic, and other operators but are all composed of the same high-level components, namely tabular data and logical expressions. A standard representation for phenotype definitions should support these formats and be modular to support localization and shared logic.

ReviewR: a light-weight and extensible tool for manual review of clinical records.

Objectives: Manual record review is a crucial step for electronic health record (EHR)-based research, but it has poor workflows and is error prone. We sought to build a tool that provides a unified environment for data review and chart abstraction data entry. Materials and Methods: ReviewR is an open-source R Shiny application that can be deployed on a single machine or made available to multiple users. It supports multiple data models and database systems, and integrates with the REDCap API for storing abstraction results. Results: We describe 2 real-world uses and extensions of ReviewR. Since its release in April 2021 as a package on CRAN it has been downloaded 2204 times. Discussion and Conclusion: ReviewR provides an easily accessible review interface for clinical data warehouses. Its modular, extensible, and open source nature afford future expansion by other researchers.

Design and validation of a FHIR-based EHR-driven phenotyping toolbox.

OBJECTIVES: To develop and validate a standards-based phenotyping tool to author electronic health record (EHR)-based phenotype definitions and demonstrate execution of the definitions against heterogeneous clinical research data platforms. MATERIALS AND METHODS: We developed an open-source, standards-compliant phenotyping tool known as the PhEMA Workbench that enables a phenotype representation using the Fast Healthcare Interoperability Resources (FHIR) and Clinical Quality Language (CQL) standards. We then demonstrated how this tool can be used to conduct EHR-based phenotyping, including phenotype authoring, execution, and validation. We validated the performance of the tool by executing a thrombotic event phenotype definition at 3 sites, Mayo Clinic (MC), Northwestern Medicine (NM), and Weill Cornell Medicine (WCM), and used manual review to determine precision and recall. RESULTS: An initial version of the PhEMA Workbench has been released, which supports phenotype authoring, execution, and publishing to a shared phenotype definition repository. The resulting thrombotic event phenotype definition consisted of 11 CQL statements, and 24 value sets containing a total of 834 codes. Technical validation showed satisfactory performance (both NM and MC had 100% precision and recall and WCM had a precision of 95% and a recall of 84%). CONCLUSIONS: We demonstrate that the PhEMA Workbench can facilitate EHR-driven phenotype definition, execution, and phenotype sharing in heterogeneous clinical research data environments. A phenotype definition that integrates with existing standards-compliant systems, and the use of a formal representation facilitates automation and can decrease potential for human error.

Generating and Reporting Electronic Clinical Quality Measures from Electronic Health Records: Strategies from EvidenceNOW Cooperatives.

BACKGROUND: Electronic clinical quality measures (eCQMs) from electronic health records (EHRs) are a key component of quality improvement (QI) initiatives in small-to-medium size primary care practices, but using eCQMs for QI can be challenging. Organizational strategies are needed to effectively operationalize eCQMs for QI in these practice settings. OBJECTIVE: This study aimed to characterize strategies that seven regional cooperatives participating in the EvidenceNOW initiative developed to generate and report EHR-based eCQMs for QI in small-to-medium size practices. METHODS: A qualitative study comprised of 17 interviews with representatives from all seven EvidenceNOW cooperatives was conducted. Interviewees included administrators were with both strategic and cooperative-level operational responsibilities and external practice facilitators were with hands-on experience helping practices use EHRs and eCQMs. A subteam conducted 1-hour semistructured telephone interviews with administrators and practice facilitators, then analyzed interview transcripts using immersion crystallization. The analysis and a conceptual model were vetted and approved by the larger group of coauthors. RESULTS: Cooperative strategies consisted of efforts in four key domains. First, cooperative adaptation shaped overall strategies for calculating eCQMs whether using EHRs, a centralized source, or a "hybrid strategy" of the two. Second, the eCQM generation described how EHR data were extracted, validated, and reported for calculating eCQMs. Third, practice facilitation characterized how facilitators with backgrounds in health information technology (IT) delivered services and solutions for data capture and quality and practice support. Fourth, performance reporting strategies and tools informed QI efforts and how cooperatives could alter their approaches to eCQMs. CONCLUSION: Cooperatives ultimately generated and reported eCQMs using hybrid strategies because they determined neither EHRs alone nor centralized sources alone could operationalize eCQMs for QI. This required cooperatives to devise solutions and utilize resources that often are unavailable to typical small-to-medium-sized practices. The experiences from EvidenceNOW cooperatives provide insights into how organizations can plan for challenges and operationalize EHR-based eCQMs.

Under-specification as the source of ambiguity and vagueness in narrative phenotype algorithm definitions.

INTRODUCTION: Currently, one of the commonly used methods for disseminating electronic health record (EHR)-based phenotype algorithms is providing a narrative description of the algorithm logic, often accompanied by flowcharts. A challenge with this mode of dissemination is the potential for under-specification in the algorithm definition, which leads to ambiguity and vagueness. METHODS: This study examines incidents of under-specification that occurred during the implementation of 34 narrative phenotyping algorithms in the electronic Medical Record and Genomics (eMERGE) network. We reviewed the online communication history between algorithm developers and implementers within the Phenotype Knowledge Base (PheKB) platform, where questions could be raised and answered regarding the intended implementation of a phenotype algorithm. RESULTS: We developed a taxonomy of under-specification categories via an iterative review process between two groups of annotators. Under-specifications that lead to ambiguity and vagueness were consistently found across narrative phenotype algorithms developed by all involved eMERGE sites. DISCUSSION AND CONCLUSION: Our findings highlight that under-specification is an impediment to the accuracy and efficiency of the implementation of current narrative phenotyping algorithms, and we propose approaches for mitigating these issues and improved methods for disseminating EHR phenotyping algorithms.